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Abstract 

The ID WORLD project involves the interfacing of a compiler, interpreter, debugger 
and editor mode to create an environment for the development of dataflow programs 
written in ID. It replaces the Tagged-Token Dataflow Architecture (Tl DA) Emulator as 
the foundation for the Multiprocessor Emulation Facility at the Laboratory for Computer 
Science. M.I.T. 

This thesis presents the design of ID WORLD, noting the need for such a system, and 
includes a detailed examination or each subsystem. Special attention is paid to the problems 
inherent in the old 7TDA Emulator, and how they were solved in ID WORLD. 
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Introduction 



The Compulation Structures Group (CSG) at the Laboratory for Computer Science 
(I.CS) is currently building a Multi processor Fmulation Facility (MFF[2]) to facilitate 
research and development in parallel architectures and languages. The first use olThe MFF 
involves the emulation of the Tagged-Token Dataflow Architecture (ITDA[1J) to 
demonstrate the feasibility of general purpose dataflow machines. 

To support these elforts. several programs have been written over the past several years 
to support dataflow software development. Unfortunately, these programs were not 
designed to be directly interfaced to one another. As a result, the software development 
cycle for dataflow programs was very long. A typical cycle would begin with the use of an 
editor to edit a file containing a piece of code written in a dataflow language. The Hie would 
then be written to disk, and a compiler would be invoked and told to read the source Hie, 
outputting a file containing die compiled code. Finally, an interpreter would be told to load 
the compiled Tile just produced, and only then would the original dataflow program be 
executed. When the interpreter would detect an error in the program, the user would have 
no choice but to go back to editing the source file and start the whole cycle over again. 

To someone who develops programs in. say. C or CLU, this may seem like a normal 
cycle. A Lisp Machine programmer, however, has become accustomed to a very interactive 
environment where programs may be incrementally edited, compiled, executed, and 
debugged, without ever leaving the editor. 

The goal of ID WORLD is to provide such an interactive environment for the 
development of programs written in ID [4]. the dataflow language used by the Computation 
Structures Group. With ID WORLD the typical software development cycle is as follows. 
A user enters the editor and types in an ID procedure. With a single keystroke, the 



procedure is sent to the ID compiler, and the output of the compiler is automatically loaded 
into the interpreter. Then, without leaving the editor, the user instructs the interpreter to 
execute the procedure, and when the interpreter detects an error the user simply continues 
editing the procedure until it is correct. Thus, with only a few keystrokes and without 
writing a single file. ID WOK 1. 1) interfaces the editor, compiler, and interpreter, making it 
much quicker to develop programs. 

The remainder of this thesis is organized as follows. Chapter two presents the design of 
GI'I A. the interpreter used in ID WORLD to execute dataflow programs. Chapter three 
presents the GITA debugger, noting how it differs from normal debuggers for sequential 
languages. In chapter four I describe ID Mode, a new editor mode for the development of 
ID programs. Chapter five discusses how each of the subsystems in ID WORLD 
generalizes when multiple machines are used. Chapter six presents the three abstract 
machines supported by ID WORLD. In chapter seven I presents conclusions and offer 
suggestions for future improvements to ID WORLD. Hnally, the Appendix contains the 
ID WORLD users manual. 



Chapter 2 

Craph Interpreter 
for the Tagged- Token Architceturc 



GITA. ihc Graph- Interpreter for the Tagged-Token Architecture, was designed by Ken 
Trail b and Richard Soley and was originally implemented by the author and Richard Soley. 
Improvements to GITA have been made by by David Culler. Greg Papadopouios. Andrew 
Chien. Steve Heller, and lihaskar Guharoy. 

2.1 liackground 

The first method of executing ID programs involved using a program called IDSys 
whieh took ID programs, compiled them into MacLisp. and executed them. This ID to 
LISP compiler was followed by an ID to Graph compiler, which compiled ID procedures 
directly into dataflow graphs. A TTDA Simulator was then created to execute these 
dataflow graphs. At the time it was envisioned that ID programs would be debugged on 
IDSys, and only working programs would be run on the TTDA Simulator. 

The TTDA Simulator was followed by a 7TDA Emulator, written in LISP for the 
Multiprocessor Emulation Facility (MEF[5J), which was supposed to be a much faster 
interpreter of dataflow graphs. The TTDA Emulator was eventually abandoned, however, 
partially because it failed to perform as expected, and partially because the MEF was in the 
process of changing over to using T.I. Lisp Machines instead of Symbolics Lisp Machines. 

The ID to Graph compiler was eventually ported to the Lisp Machine, but IDSys was 
not. The only way left to execute ID procedures was the TTDA Simulator. Debugging 
programs on the TTDA Simulator was very difficult because both the ID to Graph compiler 
and the TTDA Simulator were still relatively untested and full of bugs. Gino Maa 
eventually succeeded in debugging a 1200 line ID program, but only after many months of 
effort It was suggested that perhaps a version of the simulator which had only one fast 



processing element would simplify the debugging of programs. It w;is then suggested that 
the TI'DA Fmulalor could be resurrected and simplified. Finally, Ken 'I'raub suggested 
that it would be easier to create a new interpreter, called GITA. 

2.2 GITA vs. I he ITDA Emulator 

GITA is an object-oriented dataflow interpreter, which replaces the Tagged-token 
dataflow emulator on the MEE. In this section I will point out some of the poor design 
decisions in the ITDA Emulator, and show how they arc solved in GITA. 

2.2.1 Data Structures 

One of the biggest differences between GITA and the 'ITDA Fmulalor involves how 
each implements the fundamental data structures necessary for executing dataflow graphs. 

In the ITDA Emulator, tokens were represented as 32 contiguous bytes of data at some 
offset into a large array called the token store. Accordingly, whenever a part of the 
Emulator had a reference to a token, all it really had was an integer specifying the offset into 
the array. This made it very difficult to debug the TTDA Emulator because it was 
impossible to tell if a variable was referencing a token by just looking at its value. The user 
would have to somehow know that if the value was an integer, then it might be referencing a 
token. In addition, because the data inside a token was represented as a series of bytes, it 
was necessary to write encoding and decoding functions for each of the slots in the token 
data structure. Furthermore, there needed to be debugging functions which would take an 
olTsct into the token store and print a human-readable representation of the contents of the 
token. 

These problems were avoided in GITA by simply representing a token as a structure of 
type token. By making a token a distinct, recognizable object in GITA. it is much easier to 
debug code since there is no longer any confusion about what a variable actually references. 
Also, since a token is made up of several slots which can each contain a reference to any 
LISP object, there is no need to perform any encoding or decoding of data GITA. in fact, 



maps the dala typos in ID directly into the primitive data types of LISP, where possible, and 
where not possible a new LISP dala type is created to represent the ID data type. For 
example, the ID data types boolean, integer, and real number, arc directly represented in 
LISP by boolcans (t and nil), integers, and floating-point numbers. Since l-Structurcs 
don't map directly to any primitive LISP object, however, a new data type was defined to 
represent them. 

One disadvantage of representing the data structures in GITA as objects is that the 
object representation takes up considerably more storage than does the scheme used by the 
TI'DA Hmulator. Kor example, in the Tl DA Emulator the instructions for a dataflow 
graph were represented as an array of 8-bit bytes, and a single instruction was represented as 
an offset into this array. On average a single instruction was 8 bytes in the TI'DA Emulator. 
In GITA. however, an instruction is an object of type Instruction, and takes up about 48 
bytes of storage, a 600% increase. The reason for this large increase in size is that the 8 bytes 
in the I I DA Emulator actually encode about 12 different pieces of information. In GITA 
each piece of information is given its own slot in an instruction object, and each slot is large 
enough to reference any other LISP object. On the LISP machine this translates into 4 
bytes per slot, for a total of 48 bytes. 

In GITA. this inefficient use of storage is justified for two reasons. First. GITA is 
running on machines with large virtual address spaces. There is no need to waste a lot of 
effort saving space if you have more than you can possibly use already. And second, as a 
result of the inefficient use of storage, GITA can execute dataflow graphs considerably 
faster than the 1TDA Emulator. At most, the 7TDA Emulator could execute 
approximately 250 dataflow instructions per second. GITA. on the other hand, currently 
executes dataflow instructions at the rate of 2.500 per second. 1 This increase in speed is not 
surprising since GITA has to do almost no encoding or decoding of data before using it. 
The TTDA Emulator needed to decode the bit patterns of each instruction each time it 
interpreted it. In GITA. the bit patterns are decoded once at load time and are stored in a 
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more easily accessible, albeil more verbose, format. In sum. a belter lime/space tradeoll' 
was made in GITA than in the ITDA Emulator. 

2.2.2 F.rror Handling 

Another difference between GITA and the ITDA Emulator involves how each 
approached the problem of error handling. When either interpreter is executing a dataflow 
graph, it must make sure to catch any run-time errors, such as overflow, underflow, 
division-by-/cro. etc.. and report them to the user. There are essentially two ways to 
approach this problem: eager error handling or lazy error handling. 

The ITDA Fmulator Ux>k the eager error handling approach. What this means is that 
before it would execute any instruction, it would make sure thai no errors were going to 
happen. For example, before performing a division, the ITDA Emulator would check that 
the denominator was non-zero. It would then check for the possibility of positive and 
negative overflow, and positive and negative underflow. Only if it was sure that the division 
would not cause any of these errors would it finally go ahead and perform the division. This 
approach is analogous to the strategy of touching all the pages that an instruction in a virtual 
memory system is going to access, and only after all the pages are guaranteed to be in 
physical memory is the instruction actually executed. 

In GITA. the lazy error handling approach is taken. What this means is that GITA does 
not bother checking the arguments to an instruction before it interprets it It just assumes 
that the arguments are of the correct types and that no error is going to happen. Most of the 
time, this turns out to be a correct assumption. There are times, of course, when the 
assumption is incorrect, such as when a dividc-by-zero error happens. But for this case 
GITA just lets the Lisp Machine error handler catch the error, and intercepts it before the 
[JSP debugger is invoked. It then figures out which part of the machine got the error (the 
ALU in this case), and records any relevant information (such as the arguments to the 
instruction) so that it can explain and analyze the error at a later time. Finally, it causes the 
instruction to abort and goes on to the next one. This approach is analogous to being able to 
back out of the execution of any instruction when a page-fault occurs in a virtual memory 



system. In GITA ihis is always possible because the eommii point in i| lc execution of 
dataflow instructions occurs when a token carrying the answer is actually transmitted to the 
next instruction, and this doesn't happen until the answer is computed. If an error doesn't 
happen while the answer is being computed, then it isn't going to happen. 2 

The advantages of lazy error handling over eager error handling arc clearly 
demonstrated by examining how the ITDA Emulator and GITA each execute a floating- 
point multiplication instruction. In the Tl DA Emulator, execution of a single floating- 
point multiplication instruction required 3 function calls. 4 floating-point relational 
operations, 2 floating-point divisions, and 1 floating-point multiplication. The 4 relational 
operators and 2 floating-point divisions were done to make sure that the multiplication 
would not overflow. GITA. however, executes the same instruction with just 1 function call 
and 1 floating-point multiplication. Should the multiplication overflow the LISP system 
would raise an exception, which would be caught by GITA. 

2.2 J The Basic TTDA Kmulator Maw 

The previous two design (laws in the TTDA Emulator were actually a result of a more 
fundamental problem in its design: it was designed to behave too much like a real Tagged- 
Token dataflow machine. This is ironic since the TTDA Emulator was supposed to be a 
"soft" implementation of the machine. It was hoped that by emulating the machine in 
software, the flaws in the design of a real machine could be ironed out before it was actually 
built. The problem, however, was that the Emulator became so unmanageable that even the 
smallest changes to it were extremely hard to make. In effect, the Emulator became almost 
as unmodifiablc as hardware would have been. 



Assuming, of course. ih;it ihe transmission of the answer does not cause an error. Unless GITA itself has a 
bug in it. the transmission will always succeed. 
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2.3 The Structure of GITA 

This section presents the internal organization of GI'I'A. It assumes the reader is 
familiar with the Tagged-Token Dataflow Architecture, and the dataflow language ID. 

The internal organization of GITA is shown in figure 2-1. GITA can receive three 
different types of input, each of which is placed in its own queue. During each cycle, GITA 
removes and processes one entry from each of these three queues. We examine each of 
these queues individually. 

2.3.1 The Token Queue 

The token queue contains tokens which have left the instruction which created them, but 
have not yet arrived at their destinations. That is. it contains all the tokens still riding on the 
arcs of the dataflow graph (but not including those which are sitting in the waiting-matching 
section). 

GITA processes a token from the token queue as follows. It first looks to sec if the 
instruction is unary. If so. then it sends it directly on to the ALU section, bypassing the 
waiting-matching section. Otherwise, the token is binary. In this case the waiting-matching 
section is searched for the partner of the token. If a partner is not found, the token stays in 
die waiting-matching section until his partner arrives. If a partner is found, then it is 
removed from the waiting-matching section and both tokens are forwarded to the ALU 
section. 

The ALU section looks at the tags on the tokens to find out what operation must be 
performed. This operation will cither be an arithmetic operation, an l-Structure operation, 
or a Manager operation. In the case of an arithmetic operation, the ALU simply takes the 
two data values riding on the tokens, computes a result, packages it into a token, and sends 
the token to the destinations of the instruction it just executed. In the case of an l-Structure 
operation, the ALU takes the necessary information from the two tokens and outputs an 
l-Structure request. A Manager operation is similar, except that the ALU outputs a 
Manager request 
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2.3.2 The l-Structurc Requests Queue 

The l-Slructurc Requests Queue contains unprocessed requests for Hie l-Siructurc 
controller. GITA processes a request from ihis queue as follows. It looks at the request- 
type Held of the l-Structure request object to figure out what operation needs to be 
performed. This can be either l-helch, l-Storc. Increment Reference Count, or Decrement 
Reference Count 

In the case of an [-Ketch, the rest of the request will contain a reference to an I- 
Siructurc. an index, and a destination. The destination specifics where to send the token 
containing the value to be read. The l-Slruclure controller first looks to see if the given slot 
in the l-Structurc has already been written. If so, it takes the value stored there, packages it 
into a token, and sends it to the destination. If me slot is empty, then the l-Kctch is said to 
be deferred, and the request is suspended until a write of the slot occurs, at which time the 
request will be satisfied. 

In the case of an l-Store, the rest ofthe request will contain a reference to an l-Structure. 
an index, a value to store, and a destination. The destination specifics where to send an 
acknowledgment token, which declares that the l-Storc has taken place. The l-Structure 
controller looks to see if the given slot in the [-Structure has already been written. If so, an 
error is signalled since an [-Structure slot may be written only once. Otherwise, the 
controller sees if there are any l-Ketches waiting for the value in the slot about to be written. 
If so. it packages the data value into a token and sends it to the destinations given by each 
deferred (-Fetch. Finally, the value is stored in the slot, and an acknowledgment token is 
sent to the destination specified by the current l-Store request. 

In the case of Increment Reference Count, the rest ofthe request will contain a reference 
to an l-Structure, a number specifying the amount by which to increment the reference 
count, and a destination. The destination specifics where to send an acknowledgment token 
after the reference count is adjusted. The [-Structure controller simply increments the 
reference count by the given amount and sends the acknowledgment token to the 
destination. Decrement Reference Count is done in exactly the same way, except that the 
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reference count is decremented. 

2.3.3 The Manager Requests Queue 

The Manager Requests Queue contains unprocessed requests for the System Manager. 
GITA processes a request from this queue as follows. It looks at the request-type Held of 
the Manager request object to figure out what operation needs to be performed. This can 
be cither Invoke or Terminate. 

In the case of Invoke, the rest of the request will contain a reference to a procedure 
object, an l-Structurc Descriptor (ISD) for the arguments, an ISO for the Results, and a 
reference to the context which is invoking this procedure. The manager creates a new 
context for the invocation of the procedure, fills it with the ISD's for the arguments and 
results, and stores the reference to the caller context in the new context. It then creates a 
token and sends it to the first instruction in the new procedure, causing the procedure to 
begin execution. 

In the case of terminate, the rest of the request will contain a reference to the context to 
be terminated. The manager then releases any resources used by the context and sends a 
signal to the caller telling it that one of its sub-procedures has terminated. 

2.3.4 Beginning and Ending 

The previous three sections describe what GITA is doing when it is running, but how 
does the execution of a dataflow graph start and end? 

To begin the execution of a procedure, GITA simply creates a manager request which 
asks that the procedure be invoked with a certain set of arguments. This manager request is 
simply placed into the manager request queue and the main cycle of GITA is started. The 
system manager will then process the request, placing the initial token to be dropped into 
the dataflow graph of the procedure into the token queue. From then on GITA continues 
processing each queue until there is nothing left to do. 
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The interpretation of'ihc dataflow graph is said to have ended when there are no more 
entries on any of the Manager Request. I-Siructure Request, or Token queues. Normally 
this means that there are no more tokens in the waiting-matehing scetion. and no suspended 
(i.e. deferred) requests left in the l-Strueture controller. However, if a run-time error was 
detected during execution, or if the dataflow graph was not well formed (because the 
compiler generated bad code, for example), then there may still be tokens or suspended 
l-Slructure requests left around. When GITA finishes executing it will warn if the 
execution did not end properly. 
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Chapter 3 
The GITA Debugger 



The GITA debugger is in many ways like the normal LISP debugger. The main 
difference is that the LISP debugger allows you to look up and down a stack of frames, 
while the Gil A debugger allows you to look around a tree of contexts. * 

3.1 The F.rror Handler 

When GITA detects an error during execution of a procedure a message is printed 
saying in which part oflhe machine the error occurred. Execution continues, however, until 
there arc no more activities ready to fire. At the end of execution GITA reports the total 
number of errors it encountered. 

The GITA Debugger has commands which let you view the errors which occurred 
during the last GITA run. To debug a particular error you would type a command to the 
debugger telling it that you are interested in that error. The debugger will then set both the 
current and anchor contexts to the context in which the error occurred. 

3.2 Invocation Tree 

In GITA. a context corresponds to a stack frame in sequential languages. Whenever a 
procedure is invoked a context is created to hold its arguments and results, and any other 
information particular to the procedure's invocation. Because a procedure can execute sub- 
procedures in parallel, however, GITA must maintain a tree rather than a stack of contexts. 

An example of an invocation tree is show in figure 3-1. The root context never changes 
and is the one context which has no father. It corresponds to the top-level procedure 
invocation made which started GITA running, and is shown in the figure as context 1. The 
current context corresponds to the context which is currently being examined. As you move 
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Figure 3-1: An Invocation Tree 

around the tree of contexts, the current context is changed to reflect your position in the 
tree. In the figure, context 5 is shown as the current context Finally, the anchor context is a 
context which usually corresponds to a context at which an error occurred, although is can 
be changed to any context at all through a debugger command. This anchor context is used 
in order to facilitate moving up and down the tree of contexts, as explained below. 

3.3 Moving around the Invocation Tree 

Moving around the tree of invocations in the GITA debugger is not quite as easy as 
moving up and down the stack in the LISP debugger. A context will have at most one 
father (the caller), but may have several sons (each corresponding to a procedure invocation 
which has not yet terminated). 

Moving "up" the tree of invocations is straightforward, the current context simply gets 
set to its father. Moving "down", however, requires that a branch be selected from among 
its sons. For example, in figure 3-1 the current context is 5. Moving up the context tree 
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from context 5 would make the current context be 2. To move down, however, a choice 

must be made between the three contexts which arc the sons of context 5 -- contexts 6, 7, 

and 8. Of the three possible choices, context 6 seems the "obvious" choice. This is because 

it is closest to the anchor context. The user will probably wish to return to the anchor 

context often since it is the one at which the error being debugged occurred. Furthermore, 

it is unlikely that the user will be interested in any contexts not on the path from the anchor 

context to the root context (shown as dashed lines in figure 3-1) since they probably had 

little to do with the cause of the error. In the GITA debugger, moving down the tree of 

invocations implies moving down the "obvious" choice, if there is one. This makes moving 

around a tree of invocations in the GITA debugger just as easy as moving up and down a 

stack in the LISP debugger. 

3.4 Debugger Commands 

This section briefly describes some of the more important GITA debugger commands. 
See the Appendix for a more complete description along with a listing of all the commands. 

3.4.1 Examining the Current Context 

There are many commands in the GITA debugger designed to return information from 
the current context With a single keystroke you can get at any argument or result value, the 
ISD used to hold the arguments, the ISD used to hold the results, a local value (token), the 
current procedure object, or even the context itself. 

As it happens, some of the information is not very helpful. To someone who does not 
understand the inner workings of GITA, looking inside the context or procedure object will 
shed little light on what went wrong. In addition, all you see when you look at the local 
values in a context is a bunch of tokens carrying data headed for different instructions. The 
problem is that the current ID compiler does not provide a mapping from arcs in the 
dataflow graph to names in the source code. If it did, then it would be possible for the 
GITA debugger to provide information such as "local variable X in procedure P has the 
value Y". Instead, all that can be provided now is "some local variable in procedure P has 
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the value Y." A second version of ihc ID compiler is currently being designed which will 
output the mapping from arcs to names. As soon as it is complete, the GITA debugger will 
be much more informative. 

Probably the most important piece of information provided by the GITA debugger is 
the values of the arguments given to a procedure. If an error occurred in procedure P which 
was called from procedure Q. you can usually debug procedure P independently by editing 
it. compiling it. and then calling it directly (without going through procedure Q) with the 
arguments which caused it to fail, until the procedure works as expected. This "bottom-up" 
style of programming is facilitated by the rapid edit, compile, debug loop provided by II) 
WORLD. 

3.4.2 Backtrace Commands 

A backtrace is a listing of the contexts in reverse order starting from the current context 
and ending at the root context There are two backtrace commands which differ only in 
how much detail they provide about each context One shows only the name of the 
procedure for each context, while the other shows the name of the procedure along with its 
arguments. 

These commands are useful for figuring out where in the execution of a large program 
the error occurred. For example, suppose you were trying to debug a recursive procedure P. 
and an error occurred in some call to the procedure. If the backtrace shows that only one 
call to procedure P has been made, then the procedure probably failed in the part which was 
to do the recursion. On the other hand, if the backtrace shows many calls to procedure P. 
then the procedure probably failed in the part which terminates the recursion (the base 
case). 

Even in the case of non-recursive procedure invocations, the backtrace shows you the 
particular sequence of procedure invocations which led to the current error. This 
information is sometimes enough to tell you what went wrong. 
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3.4.3 Searching 

There is a command which will search for a context with a procedure whose name 
contains a given substring. The contexts are searched starting from the lather of the current 
context toward the root context. This is useful lor quickly jumping to some context shown 
in a backtrace. For example, if the backtrace is: 
F00 [1] <- BAR [2] <- BAZ [3] <- QUUX [4] 

Then searching Tor "BA" will make the current context be BAZ [2], and searching for "F" 
will make the current context be F00 [1]. 
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Chapter 4 
An Editor Mode for Developing II) Programs 



Both the Symbolics mid Texas Instruments Lisp Machines have a built-in editor called 
/MACS, based on the KM ACS editor. One or the features of both editors is that they have 
the concept ofa major mode. A major mode tells the editor what kind of document is being 
edited. Lvery buffer has a major mode. 

One of the most often used major modes on the Lisp Machine is. obviously. LISP mode. 
When LISP code is being edited, LISP mode tells the editor how to recognize LISP 
procedures, how to move around LISP structure quickly, how to compile LISP procedures, 
etc. 

In ID WORLD there is a new major mode called ID Mode, which defines several 
commands that recogni/e the structure of ID procedures, and interfaces the ID Compiler 
and GITA in order to simplify program development. As of this writing, ID Mode 
currently knows how to do the following: 

• It is able to move the cursor to the beginning and end of ID procedures. 

• It understands about comments in ID, and can insert and remove them at the 
end of lines of code. 

• It can send one. two, or up to an entire buffer full of ID procedures to the ID 
compiler, and will automatically load the output of the compiler into GITA. 

In the future, the following commands will be added to ID Mode: 

• Indent for ID. By hitting the TAB key. the current line of code will 
automatically be indented to the correct column. 



• 



The ability to send a batch job to the TTDA Simulator on an IBM mainframe 
directly from the editor. 
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The major advantage of II) Mi»dc is that it provides a mm of "canted panel" to Uic rest 
of 10 WORLD. Without ever leaving the edifcavtji fXKstbte to write compile execute, 
and debug ID procedures. In addition, even when the H> language k changed <to ID/83s* 
the commands in l» Mode wi« remain the some. See the appendix for a compiete 
description of the ID Mode commands. 
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Chapter 5 
ID WORM) on Multiple Machines 



When writing code for a parallel dataflow machine, one should not have to be concerned 
with the number of physical processors which will ultimately be executing his program. 
That is, if one doubles the number of processors on his dataflow machine, all programs 
should run without change to the source code. Doubling ihc number or processors should 
only have the effect of speeding up the execution of those programs. 

Similarly, one of the design goals of ID WORLD is that the user should not have to be 
concerned with the actual number of machines cooperating in the execution of his 
programs. Increasing or decreasing the number of physical processors should be 
transparent to the user of ID WORLD in that whenever ID WORLD is being used in 
multiple machine mode, each of its subsystems should automatically generalize to multiple 
machines. 

SI The GITA Server 

In order for ID WORLD to be used on multiple machines, the user requests that some 
number of additional machines be allocated to him. 3 The user's machine is known as the 
master machine, and the additional machines are known as server machines. 

In order to allow for communication between the master machine and the server 
machines, a GITA Server connection is made over the EthcrNct to each of the server 
machines. The rest of this chapter describes how each of the subsystems of ID WORLD use 
the GITA Server when running on multiple machines. 



Currently there is no good way to do this. In Lhe future we envision an ID WORLD server running on some 
machine which will be responsible for allocating and deallocating machines in the facility. 
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5.2 Gil A 

GITA am be extended to run on multiple machines in many different ways. One way is 
to map the dataflow graphs of prcx-cdurcs across several machines so that different parts or 
the graph may be evaluated in parallel, ['his is the approach taken by the T\ DA Emulator. 
While it is believed that something like (his will have to be done eventually, it would have 
required substantial changes to the single-machine version orGITA. and so a less desirable 
but more easily implemented approach was taken. 

It was decided that the procedure invocation would be the smallest unit of work shipped 
to other machines. Thus, whenever procedure P called procedure Q. GITA would decide 
where the invocation of Q should take place. Currently each machine performs round- 
robin scheduling of procedure invocations among all the server machines. The first 
procedure invocation is done on the local machine, the next on the first server machine, the 
next on the second server machine, etc. With each machine doing its own round-robin 
scheduling, the division of work tends to spread out over all of the machines in a fairly 
uniform way. 

One of the problems with having more than one machine sharing in the execution of a 
dataflow program involves the access to l-Structures. In the single-machine version, all 
l-Structures were created on the local machine and access to them was easy - just read the 
n slot. In the multiple machine version, however, it could be that an l-Structure passed in 
as an argument to a procedure was actually created on some other machine. In this case a 
request must be sent to the machine which created the structure and a reply must be sent 
back. This information could be sent over the EthcrNet. but the EtherNet would quickly 
become the bottleneck in the system. Instead, a high-speed circuit switch network is utilized 
by GITA whenever communication of information between machines cooperating in the 
execution of a program is required. 

The multiple-machine version of GITA uses the GITA Server only to start and stop each 
of the server machines. That is. the user tells the local machine to execute a procedure. The 
local machine then instructs each of the server machines to get ready to cooperate in the 
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execution of the procedure. Kadi of the server machines initializes internal data structures 
and then listens to the circuit switch network for requests to invoke procedures or read 
l-Structure slots. The local machine then executes the procedure, which will cause other 
machines to receive requests to invoke procedures, which will in turn cause more machines 
to receive requests to invoke procedures, etc. Eventually, the local machine detects that the 
initial procedure invocation has terminated, and uses the GITA Server to tell each of the 
server machines to stop listening to the circuit switch and idle until they arc needed next. 

5.3 The GITA Debugger 

When GITA is run on multiple machines, errors are trapped by each machine involved 
in the execution of the program. Whenever a machine dctecLs an error, it records enough 
information to explain what went wrong, reports to the master machine that something went 
wrong, and aborts the operation which caused the error. Because the operation was aborted 
before it could send its result to the next operation, the execution of the entire program will 
eventually come to a premature halt. At that point, each machine will be left with a certain 
number of procedures which arc only partially executed. 

When the user decides that enough errors have occurred, he enters the GITA debugger. 
At that point, the local machine must do two things: collect all errors, and build an 
invocation tree. To collect the errors the local machine simply uses the GITA Server to 
request from each machine the errors that were trapped during the last program execution. 
Later when the user picks one of these errors to debug, the current context will 
automatically be changed to the context in which the chosen error occurred. 

Building an invocation tree is a bit more tricky because the contexts still active are 
spread out over the server machines. To simplify matters, the GITA debugger uses the 
GITA Server to collect each of the active contexts on the remote machines and builds one 
large invocation tree on the local machine. After doing this, it knows the context which 
called each context (the "up" links in the tree), as well as the active sub-contexts for each 
context (the "down" links in the tree). Furthermore, it knows on which machine each 
context was invoked. 
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F.vcn when Gil A is run in single machine mode, an in vexation tree is built when the 
GITA debugger is entered. In this case all contexts in the tree will have been executing on 
the local machine. The reason lor doing this is to provide the Gil A debugger commands a 
uniform view of the world. Whenever a command requires information from a context in 
the invocation tree, a function is called requesting the information. The job of the function 
is to get the requested information anyway it can. If the context is local, then it can simply 
get the information from the data structures in the local machine. If the context is non-local 
(i.e. it was running on some other machine), then the GITA Server is used to request the 
information from the server machine. When the information is returned, it is cached locally 
so that it will not have to requested again, and is returned to the command which requested 
the information. 

Thus, not even the GITA debugger commands need be concerned with the number of 
machines involved in the execution of a program. By separating the requesting of 
information from the obtaining of the information, the GITA debugger provides an 
identical interface no matter how many machines arc in use. 

5.4 ID Mode 

The only part of ID Mode which needs to be concerned with the number of server 
machines in use is the part which sends the object code from the ID Compiler to GITA for 
loading. When running of multiple machines, each machine must be given the object code. 
This is because, unlike the TTDA Emulator, GITA does not transmit programs to server 
machines which do not have them loaded at run-time. Instead, ID Mode makes sure that all 
machines always have the latest compiled version of any procedure. 

The main reason for insisting that each machine always have the same procedures 
loaded is that it minimized the changes to GITA necessary to make it work on multiple 
machines. In the future GITA may be changed to do dynamic loading of procedures. 

Finally, a change was made to ID Mode not to support multiple machines but rather to 
exploit them. Whenever several machines are being used, and more than one procedure is 
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U) be compiled (as, for example, when an entire buffer » yuropiled), the pracedurcs are 
farmed out to each ©f ihc server machines for uwfwfoi Thus, to Umc rwpift^ to 
cojnpUca^«ttpofpmeeAMWWfreatlyrttfc«j(BA 
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Chapter 6 
he II) WORM) Abstract Machines 



ID WORLD currently supports three different abstract machines on which to execute 
dataflow programs. Each of the machines is similar in that they accept the same output 
generated by the ID Compiler, and. when told to execute a procedure with a given set of 
arguments, will each produce the same answer. The main difference involves how closely 
each attempts to simulate the actual fagged-Token Dataflow Machine. 

The first two abstract machines. GITA-I and GITA-E. are actually the same program. 
GITA, being run in two different modes. GITA-I is Gil A running in idealized mode, and 
GITA-E is GITA running in emulation mode. We treat them as different abstract machines 
because the statistics they generate differ radically and arc based on two different dataflow 
execution models. The third abstract machine, called SITA (the Simulator for the Tagged- 
Token Architecture) is implemented in PASCAL and runs on an IBM 4381. Each of these 
machines is examined individually in the remainder of this chapter. 

6.1 GITA- 1 

GITA-I is short for GITA-ldealizcd, and is realized in software by running GITA in 
idealized mode. Of the three abstract machines, it does the poorest job of simulating the 
Tagged-Token Dataflow Machine. In fact, GITA-I makes no attempt to simulate any 
realizable machine. It is best thought of as a machine which captures the abstract behavior 
of the U-lnterpreter[3]. 

The statistics collected for a program executed under GITA-I are based on the following 
assumptions: 

• All activities ready to fire are fired in the same timestep 

• All activities take one time unit to execute 
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• There is zero communications delay 

• Unbounded resources are available 

In Gl TAT a timestep is defined to be the amount of time it takes to process all activities 
which are ready to fire at the beginning of the timestep. Any activities which become ready 
to fire as a result of the firing of an instruction during timestep n will fire during timestep n 
+ 1. 

The statistics collected by GITA-I should be interpreted as measuring the inherent 
parallelism in a program. I he idealized statistics for a given program will not change when 
more machines arc used to help execute the program, nor will they be affected by real-world 
problems such as poor load-balancing, or high communications overhead. 

When a program has been debugged and is being used Tor experimentation it should be 
run once on GITA-I in order to determine the ideal performance of the program. Should 
this ideal performance show that little parallelism exists in the program, then one should not 
be surprised to find that a more realistic machine faired poorly in executing the program. 
On the other hand, if the ideal performance shows a great deal of inherent parallelism and 
yet the realistic machine failed to exploit enough of it to keep it busy, it may indicate a 
problem with the implementation of the realistic machine. 

In sum, then, statistics collected by GITA-I should be used as a control element when 
experimenting with various parameters of the more realistic machines. 

6.2GITA-E 

GITA-E is short for GITA-Emulation, and is realized in software by running GITA in 
emulation mode. Statistics collected by GITA-E come closer to describing how a real 
dataflow machine would perform on a program. GITA-E differs from a real dataflow 
machine in the following ways: 

• The minimum amount or work which can be sent to another processor is an 
entire procedure. This sometimes causes a few processors to be busy executing 
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large, complex procedures, while other processors sit idle. In a real dataflow 
machine the minimum amount of work could be as little as a single instruction. 

I-Structurcs in Gil A-H arc always mapped onto one processor. This sometimes 
causes a single processor which just happened to create a structure which is 
referenced very often (such as a table of values used by a table-lookup routine) 
to be swamped with l-Structure requests. In a real dataflow machine l-Structurc 
could be mapped across several processors, such that the first processors 
contains element 1. the second element 2, etc. 

• GITA-K is not pipelined, whereas a real dataflow machine would be heavily 
pipelined. 

• l-Structure requests and ALU operations are processed sequentially, whereas a 
real dataflow machine would process them in parallel. 

Even with these differences, the statistics given by GITA-K can be used to get a feci for 
how the setting of various parameters in a real machine might affect its performance on 
certain types of programs. For example, by changing the number of machines participating 
in the execution of a program, one can sec how well GITA-E was able to divide the work 
among the machines. In addition, it is possible to get a feel for how large certain parts of a 
real dataflow machine might have to be in order to be able to run large programs. One of 
these parts is the waiting-matching store. If die waiting-matching store is too small, certain 
programs will not be able to run. By running many different programs on GITA-K it is 
hoped that one will be able to get a better feel for the waiting-matching storage 
requirements of a real machine. 

6.3 SUA 

SITA. the Simulator for the Tagged-Token Architecture, is realized in software by 
running a PASCAL program on an IBM 4381. SITA was designed to simulate to a very low 
level the execution of a real dataflow machine. The statistics collected by SITA are very 
detailed and can provide precise information on the dynamics of a real dataflow machine. 

There are two main drawbacks with using SITA. however. The first is that, unlike all 
other subsystems of ID WORLD. SITA docs not run of a Lisp Machine. In order to use 
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SUA. a batch job must be submitted to the [MM 4381, and only some time later will the job 
be finished and the results be ready to examine. This mode of operation does not fit in too 
well with the highly interactive environment of ID WORLD. 

The second drawback is that SITA is very slow in executing programs. It currently 
executes approximately 80 dataflow operations per second, compared to GITA-E which 
executes approximately 800 dataflow operations per second per processor. 

The user of ID WORLD will have to decide for himself which is more important: faster 
execution and less true-lo-lifc statistics (GITA-E), or slower execution and very realistic 
statistics (SITA). 

6.4 Collecting Statistics in II) WOULD 

Currently it is very easy to collect and view statistics generated by GITA-I or GITA-E. 
ID WORLD has yet to be interfaced to SITA, however. 

See the appendix for a complete description of the statistics currently available, as well 
as a listing of the commands used to view them. 
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Chapter 7 
Conclusion 



ID WORLD can currently be used on any Symbolics or T.I. Lisp Machine (standalone 
configuration), or on the group of 32 T.I. Lisp Machines connected by a high-speed circuit 
switching network which comprise the Multiprocessor Lmulation Facility at the Laboratory 
for Computer Science. M.I.T. 

So far ID WORLD has proven valuable to the members of the Computation Structures 
Group who find that the highly interactive nature of ID WORLD greatly simplifies the task 
of developing dataflow programs written in ID. 

7.1 Future Directions 

ID WORLD is still a relatively new environment which is likely to change and evolve as 
more people use it. There are two places where I feel that improvements should be made 
soon. 

The first involves the handling of statistics collected by each of the three abstract 
machines. Currently there is only minor support for the viewing of statistics collected from 
GITA-I and GITA-E. and there is none for viewing statistics from SITA (at least not in ID 
WORLD). A mechanism needs to be designed which will allow statistics from all three 
abstract machines to be collected, stored, retrieved, and compared, in a consistent manner. 

The second improvement which must take place soon involves the way ID WORLD 
currently interfaces to the circuit switching network on the MEF. The problem with the 
current interface is that there is none. When one subsystem of ID WORLD wishes to send a 
message over the circuit switch it must use very low-level primitives to instruct the hardware 
to send a number of words from the current machine to some other machine. The problem 
with this approach, of course, is that there is no one uniform interface to the circuit switch 
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which everyone can use. I envision a suhstroW, m Mfce the uW MM- generic 

software substrate, being fcaptemattefi to fmwhfc a n abstruttHHt on 

top of UkchwjU switch. 
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Appendix 
ID WORLD Users Manual 



This appendix is a reference manual for ID WORLD, currently being used by ihc 
members of ihe Computation Structures Group in Hie Laboratory for Computer Science at 
M.I.T. 

ID WORLD interfaces the ID Compiler Version I. GITA. the GITA Debugger, and ID 
Mode, in order to simplify the development of dataflow programs written in ID. The ID 
Compiler Version 1 was written by Ken Traub and Steve Heller, and is based in large part 
on the ID Compiler Version by Vinod Kathail. GITA. the Graph Interpreter for the 
Tagged-Token Architecture, was designed by Ken Traub and Richard Soley and was 
originally implemented by the author and Richard Soley. Improvements to GITA hi. 
been made by David Culler. Greg Papadopoulos, and Steve Heller. The GITA Dcbugg 
and ID Mode were designed and implemented by the author. 
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Section I 
loading II) WORM) 



are: 



In order to begin using ID WORLD, you must make sure that the machine you are 
using has loaded all ofihc programs which make up ID WORLD. Currently these a 

• The Graph Interpreter for the Tagged-Token Architecture (GITA) 

• The ID Compiler Version 1 



• ID Mode 



If you arc using one of the Lisp Machines belonging to the CSG group at the M.I.T 
Laboratory for Computer Science, whether it be a Symbolics 3600 scries or T.I. Explorer, 
then ID WORLD is already loaded into your environment 

If you are on a non-CSG Lisp Machine, you can load ID WORLD by loading the file 
0AK:>lD-WORLD>ID-W0RLD.LISP. This file contains all the commands necessary to load 
all the parts of ID WORLD. 
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Section 2 
Using II) Mode 



ID Mode is a new major mode for /.MACS, the editor available on T.I. and Symbolics 
lisp Machines. It defines several commands which understand about the structure of ID 
procedures, and interfaces the ID Compiler and GITA in order to simplify program 
development. 

In ihc commands which follow, abbreviations arc used for the control and shift keys. 
They have the following meanings: V means the control key. m means the meta key. V 
means the super key. V means the hyper key. and •,„' means the shift key. A command 
such as c-sh-C is typed by holding down the control imd shift keys and typing the C key. 
Ihc command c-m-A is typed by holding down the control antl meta keys any typing A\ 



Commands such as m-X Compile File arc typed in as follows. First hold down the 
meta key and type T. In the small window at the bottom of the editor window you will be 
prompted with Extended Command:. Type in Compile File and hit <RETURN>. 

2.1 The File Attribute List 

It is a good idea to put the following line (called the attribute list) at the top of every file 
containing ID procedures that you will executing in ID WORLD: 
I -•- Mode: ID; Package: GITA -•- | 

Whenever a file which has this line at the top is loaded into the editor. ZMACS will 
automatically set the current major mode to be ID Mode, and the current package to be the 
GITA package. If you load in a file containing ID procedures which does not have this line 
at the top. then you should add it and then type m-X Reparse Attribute List. This 
will tell the editor to look at the attribute list and set the major mode and package to ID and 
GITA. respectively. 
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Alternatively, the following two editor commands can be llscd to construct the attnln.te 
list at the top of a file, or correct it if it is wrong. 

m-X ID Mode 7tJ 

c„,.. .u_ _,„• ■ r . ^.M ACS Command 

programs. ° 1 ™ i ^^ '° ' D Modc ' a modc for cdilin « »> 

m-X Set Package 

Prompt, for the name of a package to become the default for ^S 
When editing II) programs you should answer the prompt with GITA. 

After each of these commands you will be asked if you want the attribute list at the top 
ol the buffer to be updated. You should answer Yes to the question. The attribute list will 
ether be created if one doesn't exist, or the appropriate field will be updated with the 
correct information. These two commands should be invoked whenever a new .He or a new 
bullcr is created in order to correctly set up ID Mode. 

You can tell if you are in ID mode because the editor mode line (the top most line in the 
small window at the bottom of the screen) will say something like: 
ZMACS (ID) SIMPLE. ID >ARVIND> OAK: 

The symbol in parenthesis after the word ZMACS always specifics which major mode is 
currently active. If you switch to another buffer containing LISP code, for example the 
major mode becomes LISP. However, it will revert back to ID mode whenever you go back 
to editing a buffer containing ID. 

You can also tell which package you are currently in by looking down around the center 
of the very bottom line on the screen. There you will find a symbol with a colon after it 
Th,s symbol tells you what the current package is. When you are editing ID programs and 
are ,n ID mode you should see GITA: down there. While a procedure is being compiled 
you w.ll see it change to IDV1:. What this means is that the ID Compiler Version 1 is busy 
compiling the procedure. When it finishes, the package will automatically be reset to 
GITA:. 



37 



2.2 Commands in II) Mode 

There are a few special commands in ID Mode which "understand" ID procedures. 
When the following commands talk of ihc current procedure what they mean is the ID 
procedure on which the cursor is currently placed. If the cursor is between two procedures, 
then the earlier one is said to be the current one. 

ka .1 . Z\f ACS Command 

Moves the cursor to the beginning of the current procedure. That is. the cursor is 
moved so dial it is on die 'p' of the keyword -procedure. 

y A „ nc , ,, , , ZM ACS Command 

Moves the cursor to the end of the current ID procedure. That is. the cursor is moved 
just past the last character in the current procedure. 

c„,„ ,,,„ r „ • , . , ZM ACS Command 

bets the region to be the current procedure (marks the current procedure) That is 
the point is set to the beginning of Hie current procedure, and the mark is 'set to the 
end. See the ZM ACS manual for an explanation ofpoints, marks, and regions. 

The following commands are used to edit comments at the end of a line of code. 

c™ * 

ir f L^ . ,. Z MACS Command 

It the current line contains a comment, the cursor is moved to the beginning of the 
text of the comment. Otherwise, a comment is started on the current line When a 
comment is started, the cursor is moved to the comment column (a horizontal position 
where single-line comments are started by default), and beginning and ending 
comment characters are automatically added around the cursor. 

' ,f tha ... ZM ACS Command 

it the current l.nc contains a comment, this keystroke removes it Use this if you 
accidently type c- ; by mistake. 



The following commands are used for compiling procedures, regions, buffers, or entire 

files of ID code. Except for m-X Compile File the object code from the compiler is 

automatically loaded into GITA. If the ID Compiler detects a syntax error while compiling 

a procedure a message will be printed in the typeout window (a window which grows down 

over the text in the buffer), and the compilation of the procedure with the error is aborted. 
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Oihuw.sc the ci.rrc.il procedure is sou to the II) Compiler. The object code for e.ch 
pnx.-cdi.rc successfully compiled is automatically loaded into CITA. 

m-X Compile Buffer 7 

c !S h „rT rc in ,hc a,rrcnl b, " rcr is * nt l0 lhc m c '™" il "- S« c ~ 

each procedure succcsslully compiled is automatically loaded inloGITA. 
m-X Compile File 

>'romp, 5 lor a nie to compi.e. and .ends the entire lilc ,o ,hc ll/cllpn^Znle 

co lc into oi I A. Instead, a .CMC lilc is generated which contains ihe obicct code 
output by the compiler. Usc-X load Fll. to.oad .he .CMC file inloGHA 
m-X Load File 

OTA Z IT I' n " ? Tr ^ ' hC " IC "" " NMC ° r CMC «'^>""^~ 
CI I A loi loading. See the function LOAD-NMC below. 

2.3 Restrictions in II) Mode 

There are two restrictions imposed by ID Mode on the way ID code is organized in a 
Mle. Wh.le the ID Compiler docs not care about either of these restrictions, they must be 
adhered to whenever using ID Mode in order for it to work correctly. 

The first restriction is that the keyword procedure at the beginning of each ID 
procedure must be Hush with the left margin. That is. the p in procedure must always be 
m column 0. ID Mode uses this fact in order to find the beginning and end of ID 
procedures without having to perform lexical or syntactic analysis. 

The second restriction has to do with the way in which procedures may be commented 
out. As far as the ID Compiler is concerned the following is a correct way of commenting 
out the procedure f oo: 

I 

procedure foo(x) 



x • x 
I 



As a result of the first restriction, however. ID Mode sometimes gets confused as to 
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whether procedure foo is "inside" or "outside" of a comment. The only way io know for 
sure is to scan forward from the beginning of the current buffer until you get to the 
procedure in question. Hut this .scanning can be very time-consuming, so instead ID Mode 
requires that the procedure be commented out as follows: 



procedure foo(x) 
x • x I 



The idea is to make sure that the keyword procedure docs not begin in column if it is 
commented out. When uncommeniing out the procedure you must remember to once 
again position the keyword procedure so that the p is in column 0. 
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Section J 
.oiiding Compiled Procedures from the II) Compiler 



"clone you can execute an ID procedure you must first compile it using the ID 
Compiler, and then tell GITA to load the output of the ID Compiler. This can be done in 
several ways. 

The easiest way is to load a file containing ID procedures into /MACS and use ID 
Mode commands to compile one. two. or even all of the procedures in the buffer As 
explained above. ID Mode will arrange for the given procedures to be sent to the ID 
Compiler, and will automatically load the output of the ID Compiler into GITA. 

iryou already have a file which contains compiled ID procedures (which you can get by 
using the command .-x Compile File in ID Mode), then you can use the ID Mode 

command .-X Load File, described above, or the function load-nmc to load this file into 
GITA. 

(load-nmc pathname &opt1onal silent?) F 

should * «' 0f ' h " om P il «' 'D Prunes in the ffl,p«w. paZ'Z 
should have a .CMC or .NMC extension, irno extension is provided, then afile whh 
extension CMC .s looked for. followed by a file with extension .NMC I (Ten, is 

Ty "re Sed Pr0C ° dUre namCS C0 " tamed in "* ' llC wi " "°< * "n«3t,; - 
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Section 4 
I he Vhippiny of II) Procedures to I, ISP Procedures 



When GITA is told to load a Hie containing compiled ID procedures, it creates a 
structure known as a POBJ (procedure object), which contains all of the necessary 
information to allow it to interpret the procedure. In order to make it easy to execute these 
procedures. GITA creates a LISP procedure corresponding to each ID procedure loaded. 
The LISP procedure has the same name and takes the same number of arguments as the 
corresponding 113 procedure. When called, the LISP procedure tells GITA to interpret Lhc 
ID procedure with the given arguments, and when GITA finishes executing, the LISP 
procedure returns the results of die ID procedure as multiple values. 

For example, suppose we wish to execute the following ID procedure: 

procedure factoHal(n) 
(1f n - then 1 

else n * factor1al(n - 1)) 

One way to do this is to enter ZMACS. tell it to set the mode to ID Mode and the 
Package to GITA (as described above), type in the above procedure, and use the ID Mode 
command c-sh-C to compile and load this procedure into GITA. After doing this. GITA 
will create an internal representation (a POBJ) of the ID procedure factorial, along with 
a LISP procedure like the following: 

(defun factorial (n) 
(g1ta:run-code-block 'factorial n)) 

You can then call the LISP procedure factorial with an argument, which will cause 
GITA to interpret the ID procedure by pushing tokens around the dataflow graph stored in 
the POBJ. until the answers came out the bottom of the graph. The answers which come 
out of the graph will be the values returned from the LISP procedure. 

Note that this mapping of ID procedures to LISP procedures allows you to mix both ID 
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cuicl LISP. For example, if you lypc: 

(+ (factorial 3) (factorial 4)) 

to a Lisp Listener, you will cause GITA to be run twice, once for each call to factorial. 

and then LISP will compute and return the sum ofthc results of the two ID procedures. 

Furthermore, you can take the result of one ID procedure and use it as an argument to 
another ID procedure. For example, the following is perfectly legal: 
(factorial (factorial 5)) 

It will simply cause GITA to be run twice, first to compute the factorial of 5. and then again 
to compute the factorial of 120. 

You can even call ID procedures from LISP procedures (but not the other way around!). 
The bottom line is that you are free to treat ID procedures just like LISP procedures. As far 
as the user of ID WORLD is concerned, there is no difference. 
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Section 5 
resting II) Procedures 



II" the only reason you wish to execute an ID procedure is to know the result, then the 
previous section has already explained how to do this. All you have to do is get to a Lisp 
Listener, call the LISP procedure corresponding to the II) procedure, and wait for a result 
to be returned. 

When writing II) programs in ZMACS you will probably want to be doing this quite 
often to debug procedures as you write them. Whenever you write an ID procedure and 
wish to test it. you should use the ID Mode command c-sh-C to compile and load the 
current procedure into Gil A. The next thing to do is find a place where you can invoke the 
ID procedure via its LISP procedure. One place to do this is in a Lisp Listener. You can 
select a Lisp Listener (by typing <SELECT>-L or <SYSTEM>-L on Symbolics or T.I. Lisp 
Machines, respectively), execute the procedure, and then return to the editor. 

If you are already in the editor, however, a convenient place to quickly test an ID 
procedure is in the editor typeout window. The editor typeout window is a window which 
"grows" down over the text in the buffer, and behaves just like a Lisp Listener. On a 
Symbolics Lisp Machine you can get to the editor typeout window by typing the 
<SUSPEND> key. On a T.I. Lisp Machine you would use the <BREAK> key. Once the editor 
typeout window is exposed, you can type any LISP form, such as the LISP functions which 
execute the ID procedures you have just written and compiled. When you wish to go back 
to editing your programs, just type the <AB0RT> key until the typeout window goes away. 
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Section 6 
The Gil A Frame 



After you have finished using ID Mode to write, compile, and test your programs you 
should select the Gil A Frame to execute your ID programs. You can select the GITA 
Frame by typing <SELECT>-G on a Symbolics l.isp Machine «SYSTEM>-G on a T.I. Lisp 
Machine). 

6.1 Organization of the GITA Frame 

When the GITA Frame first comes up. it is divided into three sections. The top section 
is the profile pane, and is used to draw parallelism profiles (described below). About two- 
thirds of the way down the screen is a menu with items such as Loaded Execute. Finally, 
the bottom portion of the screen is a Lisp Listener which you can use to execute ID 
procedures via their LISP functions. 

There arc actually two configurations which the GITA Frame can be in. The first, which 
is what you see when you first select the GITA Frame, is called die profile configuration 
because a large portion of the screen is reserved for the displaying of parallelism profiles. 

The second configuration is called the debugger configuration. When the GITA Frame 
is in this configuration, the menu will be all the way at the top of the frame, and the rest of 
the frame will be the Lisp Listener. This configuration is used whenever the GITA 
Debugger is entered, since it more convenient to use the debugger in a large window. 

6.2 The GITA Frame Menu Items 

This section briefly describes each of the items in the GITA Frame menu. 

Load 

n mmnlr (. r , , . . GITA Menu Item 

Prompts for a file to load, then calls the function load-nmc with that file. 
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Execute civa u i, 

(ill A Menu Item 

Pops up a menu of all loaded II) p meed u res. Click on one to execute it. You will be 
prompted in the Lisp Listener lor each argument required by the procedure. GITA 
will then execute the procedure, and prim the results in the Lisp Listener. 

GITA Debugger GITA Menu Item 

Changes the configuration of the GITA Frame to the debugger configuration and 
starts up the Gl TA Debugger. See below lor a description of the GITA Debugger. 

Show Profile rrr . ., 

n . . (.'I I A Menu Item 

Changes the configuration or the GITA Frame to the profile configuration Then 
pops up a menu of all the profiles which can be shown. Click on one to cause the 
prolilc be drawn in the prollle pane. See below for a description of profiles. 

6.3 Collecting and Viewing Statistics 

So far the only reason for using GITA was to execute an ID procedure. But GITA can 
also be used to collect and view various statistics. This section describes how to tell G ITA to 
collect these statistics. 



6.4 Idealized vs. Emulated Statistics 

When GITA collects statistics it does so cither in idealized mode or in emulation mode. It 
is important to understand the difference between these two modes since the same profile 
can look very different depending on which mode was used to generate it 

The main difference between idealized mode and emulation mode is in how each 
defines the term timestep. In emulation mode a timestep is simply a certain fixed amount of 
time, such cis 2 seconds. In idealized mode, however, a timestep varies in the amount of 
time it takes to complete. It is based on the following assumptions: 

• All activities ready to fire are fired in the same timestep. 

• All activities take the same amount of time to execute. 

• There is zero communications overhead. 

• Unbounded resources are available. 
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l-mulalion mode measures how GITA actually performed in cxccuiing some ID 
procedure. If the timestep were 1 second, for example, then the first simple in the ALU- 
operations profile (described below) gives the number of ALU operations which llrcd 
during ,he first second. The next sample gives the number of operations which fired during 
the second second, etc. 

For idealized mode, however, the ALU-operations profile is interpreted as follows. The 

first sample indicates how many ALU operations fired when the proccdnrc was first started 

(01 1 A arranges for there to be one operation ready to fire at the beginning - it drops the 

first token into the dataflow graph to get things started.) The second sample indicates how 

many ALU operations fired during the .second timestep. That is. all those operations which 

became ready to fire as a result of operations firing in the first timestep arc all fired in the 

second timestep. An informal definition of a timestcpin idealized mode is. fire all and only 

those operations which are ready to fire at the beginning of each timestep. 

The following functions control which statistics mode GITA is in. 
(emulation-mode) 

Tells GITA to collect statistics in emulation mode. FUnCti ° n 

( Ideal 1zed-mode) 

Tells GITA to collect statistics in idealized mode. function 

(no-stats) 

Tells GITA not to collect any statistics. Function 

6.5 Statistics in GITA 

This sections describes the statistics which can be collected while an ID procedure is 
being executed by GITA. Note that the term timestep has a different meaning depending 
on which statistics mode GITA is in. 

ALU Operations Profile 

timestep* " Profi ' e ° f ** ni "" b ° r ° f ALU ° Perati0nS Which were fircd ""^"S 
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Walt-Match Profile 

. Statistic 

Collects a profile of the number of tokens which were in waiting-matching sections at 
the end ol each limestep. 

Invocations Profile c, .■ . 

/- ii .-I i- , Statistic 

Collects a profile of the number of procedure invocations which occurred during each 
timestcp. 

Terminations Profile <-. . 

Collects a profile of the number of procedures which terminated during^each 
timestcp. b 

I-Fatch Profile . . 

/- n . ,-. - . Statistic 

Collects a profile of the number of l-Structure fetches were done during each 
timestcp. 

I-Store Profile „ . . 

Collects a profile of the number of l-Structure stores were done during each tinitttep' 

Deferred-Reads Profile ^ 

Collects a profile of the number of l-Structure fetches which were deferred du'rina 
each timestcp. B 

l-Structure Storage Profile ^,,c 

Collects a profile of the amount of l-Structure storage in use at the end of each 
timestcp. 

The following statistics arc only collected when in emulation mode. 
Queued Tokens Profile S( . . 

Collects a profile of the number of tokens in the token queue at the end ofVach 
timestep. 

Active Code-blocks Profile Stati t 

Collects a profile of the number of code-blocks which were active at the end of each 
timestcp. 

The following statistic is only useful when in emulation mode and running on more than 
one physical processor. 

Idle Profile „ . . 

Collects a profile of the amount of time each PE was idle per timestep. 
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6.6 Viewing a Statfek 

Alter setting the atatfatacs mode and executing « 
statistics by clicking m She* ftgjffc in the 0ITA 
Thcpmfflc*^6#nfc*«nir*acai&Bli|an* 



:. you can view any of the 
eting a ttatiatfc to view. 



Section 7 
Using the GITA Debugger 



fhc GITA debugger is in many ways like ihc normal MSP debugger. The main 
deference is that the LISP debugger allows you to look up and down a stack of frames, 
while the GITA debugger allows you to look around a tree of con texts. 

7.1 GITA Debugger Definitions 

In GITA. a context roughly corresponds to a stack frame. Whenever a procedure is 
invoked a context is created to hold its arguments and results, etc. Because a procedure can 
execute sub-procedures in parallel, however. GITA must maintain a tree rather than a stack 
ol contexts. 

In the description of the GITA De bu8ger which fo||owSi tne cumn( ^^ ^ ^ 
context which is currently being examined. As you move around the tree of contexts the 
current context is changing to reflect your position in the tree. The root context never 
changes and is the one context which has no father. It corresponds to the top-level call you 
nn.de when you fir* told GITA to execute an ID procedure. Finally, the anchor context is a 
context which usually corresponds to a context at which an error occurred, although is can 
be changed by using the c- . debugger command. This anchor context is used in order to 
facilitate moving up and down the tree of contexts. See the section on Movement 
commands, below. 

7.2 Invoking the GITA Debugger 

There are two ways to enter the GITA Debugger. The first has already been described - 
when in the GITA Frame, you click on the menu item GITA Debugger. Alternatively, you 
can use the following function from wherever you were when you executed an ID 
procedure - the editor typeout window, for example. 
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(glta-debugger) 



Invokes ihc gila debugger, making ihe current eontcxi be the 



I unction 



root con tcxL 



7.3 Gil A Debugger Command Loop 

After »,*, the ,ta Debugger the root come, will he displayed, induing u,e 
name of ihc procedure and its arguments, and the c, lrS or will be to the right of a right- 
arrow, the CITA Debuggers prompt. Whenever the cursor is just to the right of the right- 
arrow, you are at top-level in the debugger. Whenever you are at this level, there are several 
things you can do. 

• You can hit the <AB0RT> key to exit the debugger. 

. You can type one of the GITA Debugger commands, described below. 

• You can type in a form to evaluate, just as if you were typing to a Lisp Listener 

%■ 

Whenever you type a form to evaluate, the command loop automatically sets the global 
ramble • to the value returned from the evaluation of the form. Thus • can be thought of 
- holdmg on to the last thing returned. Similarly, the global variable .. holds on to the 
second to Last thing returned, and ... holds on to the third to las. thing returned In the 
desenpuons of debugger commands which follow, whenever a command says that it 
"returns" an object, the variable . can be used to refer to that object For example, after 
typing the debugger command c-A. which returns the ISD for the arguments of the current 
context, you can then type (,.t, ,„o-.rgs ., ,„ make the variable foo-.rg, hold on to 
the ISD. 

Some of the commands which follow refer to a numeric ardent. A nuraeric argument 
is a number which is typed jus, before a command is issued, usually specifying which of a 
set of „ things should be done. To type a numeric argument, you hold done one of the 
control keys (either con.ro,. mem. super, or Hyper) and type the digits of the number You 
then type the command. For example, to type the c-a-A command with a numeric 
argument of 12. you could type c-1 «-, c-„-A. The section on ID Mode explains how to 
type commands such as c-m-A. 
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7.4 Debugger Commands for Krror Handling 

When GITA detects an error during execution of a procedure a message is printed 
saying in which part of the machine the error occurred. Execution continues, however, until 
there are no more activities ready to fire. At the end of execution GITA reports the total 
number of errors it encountered. The following function allows you view the errors, 
(show-errors) 

Prints a report for each error encountered during the last GITA run The erTrJTre 
pnntcd m reverse chronological order and include the time the em>r occurred and an 



7.5 Dealing with Frror objects 

When the GITA debugger is active, the following commands let you deal with errors. 



C-E 



Prints i !kf or .,ii «„. _■ • GITA Dcbu Sger Command 

eLcu es the sho cn r counterc d «" die last GITA run. This command simply 

executes me show-errors function. 

c-m-E _ 

Without n n „ ma .; , ■ G,1A ^bugger Command 

Without a numeric argument, this command returns the current error object 
Otherwise, the error object given by the numeric argument is returned. This is one 
-ay to get your hands on the arguments to the ALU operation which failed 



w, 
c-m-G 



Causes the anchor context to become the one where a particlr efrtr^ £r£ 
must use a numeric argument to specify which error you are interested in. 

7.6 Backtrace Commands 

A backtrace is a listing of the contexts in reverse order starting from the current context 
and ending at the root context There are two backtrace commands which differ only in 
how much detail they provide about each context 

C " B Display brief bactoce, showing Lhe „an,cof *e *oJ!Z££££^ 

i^SrtliSJ * backtrace ' showi " 8 "* nameof che ^^SS 
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7.7 Kxiiniininy the Current Context 

There arc many commands in ihc GITA debugger designed to return information from 
die current context. 



c-m-A 

n 
c-A 



Returns the n* ^» in the current contc,, Use , nun,* *S^ 



c-m-V 



c-V 



Returns the isn nuvi .« h ,m ,u C/7V * Debugger Command 

Kciurns inc IM) used to hold the arguments for the current context. 

Returns iho „ih .,.,i„ , u ■ , * GW A Debugger Command 

4uZ.S^cr;l uc 6 rc, " mcd rro '" lhc currcnl """"'' Usc ■ "«»"* 

Returns the ISO used ,o s torc Utc return value, of the ^Uolli"^ ^^ 

Return the procedure object from the current come,.. G,TA n,:bu ^r Command 

not the same as the actual context objeo-a dbg-context only refers to the Tctu-il 

"S SS ^ ^ ^ ' NDEX ^ '"« G ' TA **^ ™ ^ « 
to store the repl.es Irom remote servers so that it won't have to ask again. 

7.8 Movement Commands 

Moving around the tree of invocations in the GITA debugger is not quite as easy as 
movmg up and down the stack in the LISP debugger. A context will have at most one 
father (the caller), but may have several sons (each corresponding to a procedure invocation 
which has not yet terminated). 

Moving " U p" the tree of invocations is straightforward, the current context gets set to its 
father. Moving "down" the tree of invocations, however, requires that a branch be selected 
from among its sons. To make moving down the invocation tree easier, the GITA debugger 
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will sometimes pick the branch which is considers the "obvious- choice. Taking the 
"obvious" choice is done by using the c-N command without any numeric argument If the 
GIIA debugger cannot determine an "obvious" choice, then a numeric argument 
specfymg which branch to go down must be given. The GITA debugger determines the 
obvious" choice as follows: 

' '•l?,": s r cho i ™ lcx ' has no sons - thcn i " crc is "° "■" i ° *> *•"■ «« <*■» » 

' '^ZT^ ZZ "" ° nly °" C -"■ ,hC " * C "«**»- « »W choice is 

son wiiitn is closest to Die anchor context ir none of lhc sons arc in Uic n.uh ,„ 
the anchor context, then there is no -obvious" choice. P 

By debiting the "obvious" choice, you should be able to slick with using c-P .and c-« 
and never have to specify a branch unless you wan. to go olTand inspect a context a, some 
other part of the tree. 

C " m C a r UP ,° ne COme " '° WardS ** "*'• Wi * * nun.en^umtt^T^ 
many contexts towards the caller. -"bumem. goes up mat 

m-P 



m-< 
m-> 
c-N 

m-N 
c-. 



Like «-, except that detailed information about the target ^uS^ 
Goes to the root context (the oldest in the invocation tree). ""* DebUiser Comm "«' 
Goes to the anchor context G ' TA Det "' eger Cum ™""' 

uincrwi* goes down the ir" branch, where n is specified by the numeric argument 
Like c-N except mat detailed information about the target Ziu^ST"' 
Sets the anchor context to the current context ^ "^^ C ° mMn ' 



54 



7.9 Searching 

The following commands search for a context with a procedure whose name contains a 
given substring. The contexts are searched starling from the father of the current context 
toward the root context. This is useful for quickly jumping to some context shown in a 
backtrace. For example, if you type c-B and the backtrace is: 
F00 [1] <- BAR [381] <- BAZ[8] <- QUUX [583] 

Then searching for "BA" will make the current context be BAZ [8]. and searching for "F" 
will make the current context be F00 [l]. 

c-S 

PmmnPt r „, . . , GIT A Debugger Command 

Prompts for a substnng and searches up from the current context for one whose 

cuaent U one nan,e ^^ "* ^""^ " " " ndS °" C - Ulen that ««*» "<« <* 

I ik P r-« „*™«, .u , a . i i • r GlTA Debu Sger Command 

L.ke c-S except that detailed information about the target context is displayed. 

7. 1 Other Debugger Commands 

An invocation tree shows the full tree of invocations still active at the time the last GlTA 
run finished executing. The *>ns of a context are all indented to the same column 
underneath the father. 

c-T 

C l„ GlTA Debugger Command 

Shows an invocation tree of all known contexts. 

Use these next two commands to clear the screen, in one of two ways. 

c-L 

r\ a .m *u~ ..j. . GlTA Debugger Command 

Clears die screen and displays the current error message along with the procedure 
name and arguments of the current context 

r i___ tha Bn , , ,. , GlTA Debugger Command 

Clears the screen and displays detailed information about the current context, 
including the procedure name, its arguments, results, and locals (tokens). 

These next few commands let you look inside of various objects. 

r» k u i GlTA Debugger Command 

dV^rtrp A Ulin8 PrintCd ° UL This keyStroke is ^valcnt to typing 



(DESCRIBE •). 
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c-I 

iiron.1 nrim.ih ■ . . u ■ , Cl lr A Debuxxer Command 

I ctty-prmb ihc ast thing pnn.al out. This command is especially useful for viewing 
ISIX When an ISU is pretty-printed ihc contents of the l-Struciure it references is 
printed out. 

t ■ l. ■ Gl I A Debugger Command 

Jypmg this command toggles the ISD pretty-print Hag. When the Hag is set all ISDs 
are automatically pretty-printed. 

Finally, you can get online help by typing the <HELP> key. 
<HELP> 

Mm, aooncta docription of** of Uk debugger m mn,ands" °'""^ C '""«"«' 
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Section 8 
Using GI'f'A on Multiple Machines 



GITA can be made to run on multiple machines. One or the design goals of ID 
WORLD, however, was that the user should not be concerned with the number of machines 
winch are in use. Accordingly. , D Mode, the G.TA Debugger, and die statistics have all 
been designed to be used in exactly the same way no matter how many machines are 
participating in the execution of an ID procedure. 

8.1 Setting up Multiple Machines 

Note that this section is likely to change in the near future. The interface to multiple 
machmes at the moment is neither very robust nor general, and is being redesigned 
Therefore, this section will provide only a quick introduction to some of the functions which 

are currently used to run experiments on multiple machines. 

•default -processors • 

b™Ldbv'trf ns *? or * c n " d " B * inciudins •*• toi »«■*** w*K 

De used by the local machine to execute ID procedures. 

(select-flrstn-processors n) 

&» tt* : variable .Of..,*-,,.-...,., to the names of fte m* , proceTrst 

(select-processors &rest machines) F 

to ftis'fun^!' ""'"" t " |,r0C """- » *• "" •"«*'-» given as vZ'Z 
( Initial 1ze-g1ta-serv«rs) 

(reset-gUa-serveps) 

Makes sure that fte GITA-Server process is still running on eaeh of fte machinest 

has VaHed Pr0C "" r "- *"' reCSUlbliSh * am » oio " » «* <*' *• n-S" 
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(1n1t1a11ze-network) 

^Tr^T^r^l i " , ! tofM »-'"««»«~«' "> derive ,hc connS" 

™°,u It 7 h T ' h ' S mUS ' b ° ilmc 0ncc bc, " rc Cl ™ can ccculc ID 
procedures on all the machines. 

(show-netstate) 

u'^nvIn^'in'llZu nl; ' ,i0n ° rthC """"^^ «"* Ctai < S * h fo ™ 
me uuiuiincs in "default-processors*. 

(draw-netstate) 

mc maenmes in 'default-processors*. 

' ^cZ^nSZT % -'•""-P-»«»o-. co participate in *z"z 

8.2 Using the GITA Server 

The following functions may be used to force some or all of the machines in 
•default-processors* to perform some action, 
(all form) 

Causes all processors, including the local machine, to evaluate form The resultfare 
dtscarded. Does not wait for the serve* to finish evaluating/,,™ before returning 
(all-eval form) 

Causes all processors, including the local machine, to evaluate form The results^from 
each machine are collected and returned. 

(others form) 

dS US H S H al, rr CeSSOrS 6XCept the IOCal ™ chine t0 evaluate /cm,. The resul^arT 
awarded. Does not wait for the servers to finish evaluating/*™ before returning. 

(others-eval form) 

eSUTw ProCCSS T CXCCPt thC ,0Ca ' machine ' t0 ^aluatc /om,. The result^fmm 
each machine arc collected and returned. 

(execute-on peform) 

Causes form to be evaluated on the pp machine in •default-proc.ssors^ThT 
r^teare awarded. froes not wait for the server to finish exiuu* Jorm before 
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"*» i IIWWM 



(•vtl-on /*/wm) 

Cau«cs>w to be cvatouicd on ihc *# mochtnc in •4 # f«nt- 
rcsulu arc collected aadrctamai -*w«m* 

Cau«Mfr »* kwtedon ^^ *^* w ^ u . fPwwtWf . 



'oc«t««rt*. I lie 

Macro 



59 



References 



!' ArV J n , d \ D - ^ C n ilCr - R ' A - ,annucci - V - KaLhaiK K - Pin SaM and R. E. Thomas. The 
mT, . °io B n -, !o A , r J ,lilccUirc - Laboratory for Computer Science, MIT, Cambridge, 

MA. July. 1983. (Prepared for MIT Subject 6.83s) 

Tp^I^' tbL^T ' md R - A " ,annucci - A Multiprocessor Emulation Facility. 
I ah. Rep. [R-302, Laboratory for Computer Science. MIT, Cambridge, MA. October, 

3. Arvind, and K. P. Gostelow. The U-intcrpreter. Computer /J. 2 (February 1982), 42-49. 

4. Heller, Steve and Ken Traub. ID Compiler User's Manual. Tech. Rep TR-248 
RZary 1 '^ UCtUreS ^^ Ub ° rat0ry ,br Coni P^er Science. MIT. Cambridge, MA,' 

M,t'r t^mI?, M k GenCr ^ S ° ftWare f ° r thC Emulation of Multiprocessor Architectures. 
Master I h.. Ml T Laboratory hor Computer Science, June, 1985. 



60 



